Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Education Programming Science

Automated Language Deciphering By Computer AI 109

Posted by samzenpus from the what-about-dwarvish? dept.
eldavojohn writes "Ugaritic has been deciphered by an unaided computer program that relied only on four basic assumptions present in many languages. The paper (PDF) may aid researchers in deciphering eight undecipherable languages (Ugaritic has already been deciphered and proved their system worked) as well as increase the number of languages automated translation sites offer. The researchers claim 'orders of magnitude' speedups in deciphering languages with their new system."
This discussion has been archived. No new comments can be posted.

Automated Language Deciphering By Computer AI More

Automated Language Deciphering By Computer AI

Comments Filter:
  • Universal translator, here we come!

    • Re:Sweet (Score:4, Funny)

      by Fluffeh ( 1273756 ) on Wednesday June 30, 2010 @10:51PM (#32753054)
      But will it go into your ear, or will it be injected via a syringe and live in your gut is the question?

      • Re:Sweet (Score:5, Funny)

        by Anonymous Coward on Wednesday June 30, 2010 @10:58PM (#32753088)

        Good news, it's a suppository.

      • You forgot the third option... but we need a TARDIS for that.

    • Universal translator, here we come!

      pbbbbbbttt! I prefer to stick to translator microbes, thank you very much!

    • Re: (Score:3, Informative)

      by doishmere ( 1587181 )
      Their method relies heavily on the unknown language being related to a known language by some degree. At their heart of their technique is Bayesian statistics applied to lexical and frequency analysis; for this approach to work, there must be some basis for comparison.

      • Re: (Score:1, Insightful)

        by Anonymous Coward

        I think that this is more a tool for the human deciphers than a magic tool for decipher the languages. This a great tool wen you have already obtained the key points of the language, with this you can evade the most tedious part that is going word for word to obtain the language and the reduce the necessary time for decipher it. Also with this tool is possible the case were you decipher the language but this language is wrong, but this don't mean that all of the deciphered is wrong as the most possible with

    • Re:Sweet (Score:5, Funny)

      by grcumb ( 781340 ) on Wednesday June 30, 2010 @11:18PM (#32753200) Homepage Journal

      Universal translator, here we come!

      Cool! Can I bring it into my next marketing meeting?

      • Re:Sweet (Score:5, Funny)

        by Walt Dismal ( 534799 ) on Wednesday June 30, 2010 @11:32PM (#32753272)
        Only if the gross gains in closing juncture exceed the long-term sustainability goals of the viability imperative for all mass interoperability. We at Mega Industries believe this will move us forward to our cloud-based monetization of the human-media dynamic which is strategically important in an ever-evolving mobile continuum. We have directed our customer experience champions to ensure consumers realize this when they call in with emphatic expressions of dissatisfaction.

        • Only if the gross gains in closing juncture exceed the long-term sustainability goals of the viability imperative for all mass interoperability.

          Only if we can update the UI for version 2 and sell it a second time to the same saps.

          We at Mega Industries believe this will move us forward to our cloud-based monetization of the human-media dynamic which is strategically important in an ever-evolving mobile continuum.

          When everyone has it, we can turn it into a subscription-based cash cow.

          We have directed our customer experience champions to ensure consumers realize this when they call in with emphatic expressions of dissatisfaction.

          Tell the whining losers that premium support is only available with the Platinum Care package, and transfer them to "Gord-on" in the Mumbai sales office.

        • Pffft, please, your plan is to have emphatically expressed dissatisfied consumers realize that your gross gains within the closing juncture exceed your long-term sustainability goals for all viability imperatives, which will allow the move to cloud-based monetization of the human-media dynamic? It is but a futile attempt, you may as well give up right now, no matter how much time your customer experience champions waste on a single call.

          Here, at GOD Industries, we know better than to rely on such clearly m

        • I tried running your statement through the deciphering-AI, but the process killed itself before completion. I checked the debug logs, but the weren't very helpful. Just a bunch of 'e's, 'y's, 'a's, and some 'r's and 'g's strung together.

          I...I think it was screaming...

      • Re: (Score:2)

        by oiron ( 697563 )
        Read it again: It depends on similarity to a known language...

      • He said universal translator, as in it only works on languages of this universe. Marketing speak is from the anti-matter dominant universe, as evidenced by the fact that the more it is spoken, the less is actually communicated.

      • Re: (Score:1)

        by MoriT ( 1747802 )
        There is a perfect translation system that extracts all content from marketing speech: earplugs.

  • Answers to all TFA questions (Score:5, Informative)

    by cappp ( 1822388 ) on Wednesday June 30, 2010 @10:53PM (#32753068)
    Just so we can keep the “didn’t read TFA” comments to a minimum: The four assumptions as laid out in the article are:

    - The language being deciphered is closely related to some other language: In the case of Ugaritic, the researchers chose Hebrew.

    - There’s a systematic way to map the alphabet of one language on to the alphabet of the other, and that correlated symbols will occur with similar frequencies in the two languages. The system makes a similar assumption at the level of the word: The languages should have at least some cognates, or words with shared roots, like main and mano in French and Spanish, or homme and hombre.

    - The system assumes a similar mapping for parts of words. A word like “overloading,” for instance, has both a prefix — “over” — and a suffix — “ing.” The system would anticipate that other words in the language will feature the prefix “over” or the suffix “ing” or both, and that a cognate of “overloading” in another language — say, “surchargeant” in French — would have a similar three-part structure.

    . The article also notes the success rates where it states that

    Ugaritic has already been deciphered: Otherwise, the researchers would have had no way to gauge their system’s performance. The Ugaritic alphabet has 30 letters, and the system correctly mapped 29 of them to their Hebrew counterparts. Roughly one-third of the words in Ugaritic have Hebrew cognates, and of those, the system correctly identified 60 percent. “Of those that are incorrect, often they’re incorrect only by a single letter, so they’re often very good guesses,” Snyder says.

    Critics noted that

    The researchers’ approach, he says, presupposes that the language to be deciphered has an alphabet that can be mapped onto the alphabet of a known language — “which is almost certainly not the case with any of the important remaining undeciphered scripts.” It also assumes, he argues, that it’s clear where one character or word ends and another begins, which is not the case with many deciphered and undeciphered scripts. The decipherment of Ugaritic took years and relied on some happy coincidences — such as the discovery of an axe that had the word “axe” written on it in Ugaritic.

    • Re:Answers to all TFA questions (Score:5, Insightful)

      by MichaelSmith ( 789609 ) on Wednesday June 30, 2010 @10:59PM (#32753092) Homepage Journal

      The decipherment of Ugaritic took years and relied on some happy coincidences — such as the discovery of an axe that had the word “axe” written on it in Ugaritic.

      Maybe I should go around and write "computer" in English on all my computers, as a service to future language researchers.

      • Pfft, why? (Score:5, Funny)

        by mdenham ( 747985 ) on Wednesday June 30, 2010 @11:01PM (#32753108)

        Label at least one computer "ham sandwich" to confuse future language researchers.

        Alternatively, label each computer with a character's name from (insert show of your choice here).

        • In all of the computer labs I've been to, the name of the computer is visibly displayed in front somewhere. The names of all teh computers in the lab usually revolve around a common theme, e.g. periodic table of elements, Simpsons characters, HHGTTG characters, etc.

          You better hope English never becomes extinct, because an important period in human history would be forever lost.

          • Re: (Score:3, Insightful)

            by L4t3r4lu5 ( 1216702 )
            How idiotic. Name servers that way if you must, but workstations should be named by geographic location, building, room, station number. Nicknames don't count, but for sanity's sake name your equipment logically.

            • The word you are looking for is "systematically", not "logically". And unless you're talking about a whole building's worth of computers, it's simply not worth it to indicate a location in the name, "Huey" is a lot easier to remember than "B2R22S15".

              • This guy was talking about a computer lab. I get the impression that Huey, Duey, Louie, Barney, Smarmey, Charley, Blarney, Indigo Montarney etc will get particularly bothersome to keep tabs on as a convention. Why not B2R22Cad4? CAD machine 4 in lab 22, building 2. Not easy to remember, but memory isn't required. You have all of the information you need without having to learn anything but a naming convention.

                • This guy was talking about a computer lab. I get the impression that Huey, Duey, Louie, Barney, Smarmey, Charley, Blarney, Indigo Montarney etc will get particularly bothersome to keep tabs on as a convention. Why not B2R22Cad4

                  You know, if you are the kind of person that prefers numbers over pronouncable names, computers also have IP addresses. Just use those, and leave the hostnames for the rest of us...

            • And then when it's moved to another room?
              • Good point. It's terrible that hostnames are hard-coded into the operating system on installation, and that sticky labels are permanent fixtures once applied.

                Boy, I could sure save some money replacing equipment which needed to be moved by changing the hostname and printing a new sticker.

                • It's terrible that hostnames are hard-coded into the operating system on installation, and that sticky labels are permanent fixtures once applied.

                  Funny you should mention that - I worked in a place once where that appeared to be true.

                  By trial and error I discovered what the IDs of some of the printers were and relabled them. Next day, somebody had stuck new labels over the top with the old, wrong, IDs on them.

                  • That doesn't even make sense! What the hell?! I could understand if they had problem remembering the exact printer model when selecting which area to print to and gave it a name like "Barney", but amending an identifier specifically to make supporting it easier? It beggars belief...

                    • Encroaching on territory, I guess.

                      It was full of people who wouldn't jump in a stream if their feet were on fire unless someone specifically told them to. And no, it wasn't military/defense related at all.

                • Re: (Score:2)

                  by Quirkz ( 1206400 )

                  Boy, I could sure save some money replacing equipment which needed to be moved by changing the hostname and printing a new sticker.

                  I think you're missing the point there. Yes, it's pretty easy to change a computer name, but then you also have to update all people and/or software that connect to the server as well, and that's far from trivial. Sending out a mass email to the entire company saying "we've moved the following five servers from room A to room B, so please remember to change the corresponding digits in their names whenever you use them" doesn't go over very well.

                  I've worked at a place where the server names were completel

                  • I guess there are some people who are better at remembering room numbers and some people who are better at remembering "names". I don't know the numbers of any of the rooms on my corridoor (including mine... and no I won't turn around to read it off the door) - but I do know which one is Mondrian, Monet, Magritte etc.

        • Label at least one computer "ham sandwich" to confuse future language researchers.

          You might be interested in this approximation of what it would cause: http://www.mcsweeneys.net/2010/6/10packman.html [mcsweeneys.net]

        • You laugh, but names from a TV show used to be the server naming convention at one place I worked. Makes for interesting conversations:

          "Uhura is down again and Kirk is acting up, Spock is still blocking incoming attacks."

          "Alright, I'll bring up Scotty and RedShirt to take some of the overload. Promote Sulu to be in charge until we figure out what's wrong with Kirk."

      • Maybe I should go around and write "computer" in English on all my computers, as a service to future language researchers.

        Just don't, like many would do, put your label on the monitor.

        • Re: (Score:2)

          by Quirkz ( 1206400 )
          Occasionally I'll find a pile of server faceplates (or bezels or whatever you want to call them) on the floor, in front of a stack of label-free servers that have had their faceplates removed. Talk about a waste of time trying to sort that out. Yes, I fix that situation with a labelmaker as soon as possible, but it really astounds me that anyone would think labeling a removable part (and nothing else) is the way to go.

      • Re: (Score:3, Interesting)

        by vlueboy ( 1799360 )

        The decipherment of Ugaritic took years and relied on some happy coincidences — such as the discovery of an axe that had the word “axe” written on it in Ugaritic.

        Maybe I should go around and write "computer" in English on all my computers, as a service to future language researchers.

        Extinct language researchers examining english would fail at this same task 3000 years from now. English has no nouns --it has brand names: today's "computers" have big "Dell" logos but not "Computer."

        Also, how would researchers realize that [Apple Mac Glyph] isn't an integral part of our "ancient moon runes" if seen from their era? :)

    • Re: (Score:3, Interesting)

      by DurendalMac ( 736637 )
      Darn. So the Voynich Manuscript is probably not a prime candidate.

      • Re: (Score:2)

        by oljanx ( 1318801 )
        I wouldn't worry too much about that. They Voynich Manuscript is likely the work of a madman, who used a very inconsistent cipher to encode plain text from a language he was not fluent in. Then he added several hundred little tiny pictures of naked women, and a bunch of plants he saw on some sort of "vision quest".

      • Re: (Score:3, Insightful)

        by jd ( 1658 )

        Neither is my great great grandmother's cookbook. Which really is a shame, as I strongly suspect the recipes make something more edible than what's served at the local coffee shop.

    • Re: (Score:1)

      by jhoegl ( 638955 )
      If I didnt RTFA, what makes you think Ill read your translation?
    • So this probably isn't going to help with Rongorongo, then.

      • In the case of Rongorongo, if it is a written language, then it is probably a written form of the Rapa Nui, the language of Easter Island. In any case since Rapa Nui is a polynesian language we'd be able to compare it to other Polynesian languages. However, this has already been done with no success.

        Part of the problem with Rongorongo and with other undeciphered scripts is that we don't know what counts as a distinct character, the character vs. glyph problem. It is not clear from the article if this system

        • The process they describe assumes the characters are known, alphabetic and words are somehow separated. So I guess until the characters have been separated, it won't do much good on Rongorongo.

    • The decipherment of Ugaritic took years and relied on some happy coincidences -- such as the discovery of an axe that had the word "axe" written on it in Ugaritic

      Sorry but I had to lol at this. What was actually written on the axe was "Bill" - because it was his axe. And now the deciphered writings are all containing phrases like "That duck has an axe!", "The members voted and passed the new Axe" and "Monday - Remember to pay the Axes".

      But I guess it does make sense to write "Axe" on an "Axe", just to be sure. Oh bugger - where'd I put my Axe, all I can find is my Bill.

      • Yeah if you work on a building site you engrave your name on your tools. But fire axes in my building are labelled, as are toilets and emergency exits, even though the labels are pretty obvious.

        The yarra river in Melbourne has that name because the local aboriginal people pointed to the river and said that word but it turned out later they were commenting on the rate of flow.

        • Re: (Score:2)

          by jd ( 1658 )

          It's also why an inordinate number of mountains are called "your finger, you fool" and "who is this fool who doesn't know what a mountain is?"

    • Re: (Score:2)

      by tgv ( 254536 )

      Thanks for the summary. They are really limiting preconditions, worth pointing out. Still, it's a decent achievement (that's coming from someone with a Phd in NLP). The combinatorial problem is quite huge and you need some indication that your translation makes sense. This study shows that the use of cognates and/or word structure may help. I would think it's possible to get rid of the alphabet restriction.

      But these assumptions have other possible uses too. I'd think that they could be used to find relation

  • Linear A Implications (Score:5, Interesting)

    by DowdyGoat ( 1830958 ) on Wednesday June 30, 2010 @11:03PM (#32753124)

    This is very cool for us undeciphered language fans.

    In the article, the language author Andrew Robinson correctly points out that this computer program won't work for languages that don't have a known language that is close to them, say like for Linear A found on Crete, which is definitely not Greek like Linear B turned out to be. There is a lot of speculation that Linear A is a native Minoan (Cretan) script, largely unrelated to any other known script.

    However, parallel with Linear A on Crete was a Cretan pictographic script, which may, or may not be related to Egyptian hieroglyphics. The Minoans had known trading ties to Egypt, which had written language long before them. If a relationship could be found (via this computer program) between the Minoan pictographic script and Egyptian hieroglyphs, then that might give insights into how the Linear A script was set up (which is a syllabary script).

    The only difficulty is that there may not be enough of the pictographic script to work--I'd imagine you'd need a fair number of examples to really allow the computer to compare and contrast.

    • Re: (Score:3, Informative)

      by KritonK ( 949258 )

      Actually, the program might be able to help: From what I understand, the Linear A alphabet is related to the linear B alphabet, which has been deciphered, even though the languages may be different. We know a bit about context (what we have are mostly inventories), and we even know the meaning of one word: the one next to the total of the amounts in the inventory probably means "total". Furthermore, that word, ku-ro, is similar to a form of a Greek word for "total" ("houlon"), so it is very likely that the

      • Nah, it's not gonna be much help with Linear A. Although without a solid decipherment it's hard to be sure, a majority of the characters in Linear B also appear in Linear A. There are also names that appear in both scripts. This of course no guarantee that all the symbols had the same values in both scripts, but it's a reasonable starting point.

        Furthermore, Linear A is a syllabary, not an alphabet, and they used logograms extensively. Ugaritic, being an alphabet, is much simpler. They haven't demonstrated

    • Re: (Score:2)

      by jd ( 1658 )

      Well, a more obvious implication is that if you fed in some percentage of Linear A texts and Cretan pictographic texts, you'd get virtually the same results as feeding in a different set of texts (ie: symbols should always equate to the same opposite number) if they are truly related.

      This would at least let you identify if the texts are indeed of the same language, even if you can't read it, which is further along than we are now.

    • What's the difference between linear A and perl?
      One day we might be able to read linear A ... drrrtish!

    • On the other hand it could be useful since a program could do such a test against every known language quickly as long as you rented enough CPU time.
      I imagine such a task would take a long time to do by hand.

      • Re: (Score:2)

        by jfengel ( 409917 )

        It's a good thought, and definitely worth a try once they've worked the algorithm more. (This is very preliminary stuff.)

        But Linear A is going to be hard. There are a lot of fragments, but they're still fragments. The longest texts are only a few sentences long, and most are much shorter than that.

        Nonetheless, it's a very promising start. When you combine what the algorithm can put out with the rest of what researchers know (semantic information that the algorithm doesn't have and probably won't any tim

  • Next step: (Score:2, Insightful)

    by BoppreH ( 1520463 )
    Voynich manuscript! [wikipedia.org]

    If only we could find a language that is similar enough...

    • Thats amazing. I will have to set aside some time to go through it. My guess is that the document is an attempt to create a written script for an Asian language which is only spoken. Cantonese comes to mind because speakers of that language currently borrow mandarin and chinese writing when they want to write stuff down.

      • http://en.wikipedia.org/wiki/Voynich_manuscript#Exotic_natural_language [wikipedia.org]

        It seems plausible, all the statistical and historical evidence back it up, but it's quite strange that even with this critical hint nobody has solved the mystery yet.

      • Re: (Score:2)

        by iserlohn ( 49556 )

        Cantonese is a dialect of Chinese (as is Mandarin). In fact it is more akin to Middle Chinese than modern Mandarin. It is commonly accepted that Tang dynasty poetry sounds better in Cantonese due to the more similar tonal structure. Basically, it is believed that Cantonese has gone through less changes over the (1500) years from Middle Chinese than Mandarin.

        It is similar to how it is now believed that Elizabethan English sounds more like American English than British English / Received Pronunciation. When c

    • That's interesting, I have not come across this before.

      I last worked in computational linguistics over five years ago and but when I left there were a good supply of techniques for automatically extracting meaning from an unknown text.

      My own research was able to build up both a dendrogram and word vectors from any sufficiently large corpus, and a quick google search turned up http://www.springerlink.com/content/fp17278783422256/ [springerlink.com] which shows that the field is continuing to develop. I would expect that by no

    • Re: (Score:2)

      by Trepidity ( 597 )

      The problem is that one of their four assumptions is that the script for the undeciphered language maps characters 1-to-1 onto an existing language's script in a way such that letter frequencies are similar, which is something people have already looked for and which appears not to be the case with the Voynich manuscript.

    • Wow, thanks for that one. I ended up sidetracked for hours reading about that, and trying to fathom its meaning for myself. Coolest "something new for today" I've learned in a very long time.

  • Could be handy. It is a bit limiting that it requires you pair the target language with one where you're relatively sure that the morphology and word roots are fairly similar, and that the writing systems are similar (structurally and statistically).
    I guess there might be some way to handle some possible differences in script type (comparing a language written with alphabetic system to one written using a syllabary or abjad) by producing a fake alternate writing system for the known language that would be
  • Unaided computer program != computer AI. Not even if you use Bayesian statistics. Leave the hyperbolic headlines to the common newspapers. After all, This Is Slashdot.
  • 1 b37 17 \/\/1££ |-|4\/3 pr0b£3|\/|$ \/\/17|-| |\/|¥ b3r |-|4xx £337 $p33|{

    • It probably would have problems with your leet hacker speak, but it isn't that hard to decypher. Then again, since some of the output I've had from OCR resembles your text, maybe not...

  • So, when are they going to apply this to the Voynich manuscript [wikipedia.org] ?

  • the discovery of an axe that had the word “axe” written on it in Ugaritic

    A conversation in Semitic times:
    "What's that?"
    "Dunno..." examines the object "...it says on here that it's an axe."

    • Re: (Score:2)

      by Nyder ( 754090 )

      the discovery of an axe that had the word “axe” written on it in Ugaritic

      A conversation in Semitic times:

      "What's that?"

      "Dunno..." examines the object "...it says on here that it's an axe."

      Honestly, i would think that it was the name of the person who owned it myself.

      • Honestly, i would think that it was the name of the person who owned it myself.

        IIRC there are other examples of axes with the word "axe" written on them in languages known at the time in the area, so it wasn't that big a leap.

  • They should put on-line a DB of documents that have been translated and then allow others to build a translator. In fact, if smart, they would do this as a competition in which the winner could create a new company based on it, with a large investment by Google.
    • Google has dominated the NIST machine translation competition for years before
      they stopped participating. I don't think that they need too much external help.
      • Yes, they have dominated. HOWEVER, with the approach that I just suggested, it would allow them to help move things up a notch, and give them a chance to buy a potential competitor down the road, and turn it into subsidiary.

  • Screw the article.... (Score:3, Informative)

    by djupedal ( 584558 ) on Thursday July 01, 2010 @01:02AM (#32753698)
    IBM, as one example, has been on this hard since 2002 ( http://news.cnet.com/2100-1008-998264.html [cnet.com] ) when the prize was first announced....stop going all lady gaga over stuf that is so old it can't even be recycled properly.

  • You want to impress me... (Score:3, Funny)

    by ngc5194 ( 847747 ) on Thursday July 01, 2010 @01:44AM (#32753806)
    ... see if it can decipher some of the perl code I've had to take over.

  • Iberian from Basque Language? (Score:1, Interesting)

    by Anonymous Coward

    Iberian language was spoken in Spain before the Roman Empire. It has some similarities with Basque Language. The texts in iberian are few, anyway I wonder if this language could be decoded using this tool.

  • I need something that understands the binary language of moisture vaporators.

  • If they are undecipherable languages, how do they verify the results are accurate?

    • Which is why they tested it on a deciphered language. They are making the assumption that if it is relatively accurate in one case which meets all four of their preconditions that it will be relatively accurate in more cases which meet the same preconditions. That seems to me to be a reasonable assumption.

      But also note that, at present, this tool best serves as an aid to those trying to decipher languages. The article states that the output has limitations that make it rather inutile for the general publ

    • If they are undecipherable languages, how do they verify the results are accurate?

      In general, there are two ways to test a decipherment. The first is to compare it to a bilingual text (e.g., the Rosetta Stone). Ancient Sumerian is apparently unrelated to everything else, but there were a lot of bilinguals so the decipherment is pretty firm.

      The second method is to use the decipherment to decipher a new text. For instance, the first big test for Michael Ventris's decipherment of Linear B was using it on some newly discovered tablets. Obviously there's more uncertainty with this method,

  • From TFA:

    An incidental challenge in developing a computer system that could decipher Ugaritic (inscribed on tablet) was developing a way to digitally render Ugaritic symbols.

    Riiiiight. What did they feed their software? Photographs of stone tablets?

  • SHAKA! When the walls fell. :(

  • Like this [wikipedia.org].

  • (Insert obligatory wishful thinking about the Voynich Manuscript here.)

  • ...can it read a doctor's handwriting?

Slashdot Top Deals

God doesn't play dice. -- Albert Einstein

Close